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Background 


Occupancy detection with Passive Infrared Sensor (PIR) is state-of-art today for smart lighting control 
applications. Basically, when the PIR detects the movement of human body, it sends out a signal which 
triggers light on/Off. 


There are some drawbacks with PIR occupancy detection. A PIR sensing could not provide any further 
information relating to movement of a human body. For example, is a person going into a room or is the 
person jumping up or down? Similarly, in terms of adjusting the lighting control operation (e.g., 
dimming or brightening of a light), a PIR sensor typically cannot discriminate between different types of 
movement a person may make. Therefore, if a person wants to adjust performance of lighting, the 
person must manually touch a button or a switch/dimmer bar to enable dimming variation, light color 
tuning or any other advance features. 


An image sensor is occasionally used for security purposes rather than lighting control. Drawbacks, such 
as privacy concerns and higher costs, have prevented its broad adoption to the lighting control industry. 


Gesture detection, in general, has been applied in many areas such as automotive, healthcare, and 
virtual reality. Due to its complexity and expensive design, gesture detection has not been used today 
for a simple lighting sensing and control application. 


A novel solution of Smart Sensor with gesture detection for Lighting & Control application 


Here, we introduce a novel gesture detection mechanism for smart lighting control. It could be designed 
with on-shelf high volume manufacture (HVM) components (low cost). A simple gesture detection 
scheme is proposed to enable a feature uniquely suitable to lighting applications. A Smart Lighting 
Sensor could take a full control of its operations with human gestures. All of these can be accomplished 
in a touchless manner, which is even more critical due to hygiene concerns under COVID-19. Such a 
gesture detection scheme can be implemented by integrating with a traditional occupancy detection 
sensor such as PIR sensor. This leads to a cost advantage in HVM production. It also provides the ease of 
use for an end-user. 
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1. Concept of Smart Lighting Sensor with Gesture Detection 


A conventional occupancy sensing with a passive infrared sensor (PIR) is used to monitor the 
change of heat emitted by a human body. As the variation of heat is detected due to the body 
movement, this indicates the occupancy of a person in a certain space. The field-of-view (FOV) 
of the PIR sensor may not be achievable or may be somewhat limited due to an insufficient 
thermal disturbance appearing on the PIR sensor. For example, when a person is out of direct 
sight of view from the PIR sensor (e.g., sit at office blocked by cubic wall) or if a person’s 
movement is not large or fast enough (e.g., typing on CPU keyboard), the PIR sensor may not be 
triggered. In addition, such a PIR sensor cannot recognize “how” a person moves. 


The Smart Lighting Sensor with Gesture Detection consists of three basic functional blocks as 
follows: 
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The principle of Smart Lighting Sensor with Gesture Detection is fundamentally different from 
occupancy detection with PIR sensing. Such a smart sensor operates in a manner called as Time- 
of-Flight (ToF), which detects not only human occupancy but also the “characteristic” of human 
movement, for example, one’s moving direction or travel trajectory. 


Basically, the sensing unit consists of two elements, signal transmitter and signal receiver. 
Motion of a human body can be detected with a pair of transmitted and received signals. The 
relative distance between a sensor and a subject (human) can be derived based on the time 
duration of signal travel (e.g., ToF). 
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In practice, there are multiple pairs of transmitted and received signals emitted by a single 
sensor. Therefore, the relevant information on an object in 3D space could be obtained. 


Depending on the type of signal applied in a ToF implementation, the sensor could be 
referenced to infrared vertical-cavity surface-emitting laser (IR VCSEL), ultrasonic or radio 
frequency (RF) radar. There is a two-element (one pair of transmitted/ received signals) sensor 
from multiple vendors in the market today. In addition, sensors with four or more -elements are 
currently in development. 


The functional block of “Front-end Data Processing” is by its definition to perform basic data 
collection and simple post-data manipulation. And it stores the related information in a local 
register or memory. Then, it outputs data stream on a standard wire I/O interface (e.g., I?C) or 
via a wireless protocol to a computer host such as a microprocessor. 

The data stream updates “dynamically” and instantly, representing both the position and 
direction about the movement of a human gesture. 


To reduce cost and simplify the design, an integrated sensor with both a sensing component and 
a “Front-end Data Processing” unit on a single IC package can be designed. 


The “Control Apparatus” is the “brain” to make a decision for an action. Such an action can be 
switching a relay or adjusting light dimming or tuning other performance. Typically, the 
apparatus consists of a microcontroller, power converter, relay, and other relevant analog and 
digital circuits. 


2. Principle & Operation of the Smart Lighting Sensor with Gesture Detection 
As discussed in the previous section, the Smart Lighting Sensor with Gesture Detection can 


provide information on both the position and movement of human in a space. Such information 
can be utilized as well as customized to a Lighting Control application. 


Sensing with Gesture Detection 


With a multi-element ToF sensor, different hand movements or human gestures can be 
distinguished using different sensing elements to detect movements/gestures in 
different areas/zones of space. Besides the relative position in a space, the direction of 
movement and the trajectory of a “trip” can be recognized and captured. Such 
information can be processed, and then, used to control light operations. 


Here are a couple of examples for illustration. However, this disclosure should not be 


limited to the cases provided below. 


A.) Four-zone Sensing with Gesture Detection 


As illustrated below, a sensor with four-zone detection could identify the relative 
movement across different locations, indicated as A, B, C and D. As a person makes 
a hand gesture according to a pre-defined pattern, the person can control light 


performance without physically touching a device. 
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B.) Sixteen-zone Sensing with Gesture Detection 


With more granularity of a sensor with gesture detection, more advanced features 
could be supported. 


The drawing below represents sensing zones divided into four-by-four blocks. Such 
an arrangement could be utilized and support multiple modes of operations. 
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Light Switch - ON = Move a hand up-down-up vertically 

Light Switch - OFF = Move a hand left-right-left horizontally 
Light Dimming - Increase = Move a hand from bottom to top diagonally 
Light Dimming - Decrease = Move a hand from top to bottom diagonally 
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For example, the central two-by-two block (Ao, Bo, Co, Do) can serve as a basic set of 
product features as defined in the section “A” above. Then, the remainder of the 
divided sensing zones (A. ... Lı) can be utilized to enable a more advanced function. 
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Here shows one of these applications to control light dimming on a granular level. 
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o Light dimming-down. 
= Move a hand from right to left (Di -> Ai) to signal the operation request 
for light “dimming-down” 
= Move a hand down with 1-step (A; -> E1) to enable 10% reduction in 


brightness 

= Move a hand down with 2-step (A; -> Gi) to enable 30% reduction in 
brightness 

= Move a hand down with 3-step (A1 - > l1) to enable 50% reduction in 
brightness 


o Light dimming-up. 
= Move a hand from left to right (lı -> L1) to signal the operation request 
for light “dimming-up” 
= Move a hand up with 1-step (L, -> Hi) to enable 20% increase in 
brightness 
= Move a hand up with 2-step (L1 -> F1) to enable 50% increase in 
brightness 


= Move a hand up with 3-step (L1 -> Di) to enable 80% increase in 


brightness 


C.) 3D Sensing with Gesture Detection 


To fully take advantage of Smart Sensor with Human Gesture, one can also consider 
its sensing effect on a third dimension in space. For example, a person comes closer 


or moves away from a reference point. 


For example, when a person goes out of an office (across an entry door), the light 
would turn off accordingly. Another application would be people-counting. By 
sensing a person’s movement and direction, we would know how many people 


move in and out of a space (e.g., an office in a building). 


Lighting Control Application with Smart Sensor with Human Gesture 


As discussed above, the Smart Sensor with Human Gesture could respond to a gesture 
by a user to control the performance of lighting. Based on the sensing information, the 
lighting control apparatus would process the sensing data and issue an action request 
accordingly. For example, in the case A.) discussed above, the user gesture signals a 
request of “Light-On” with hand moving up-down-up vertically. Then, the control unit 
would output corresponding commands to switch a relay to turn on the light. 
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The flowchart below illustrates a procedure on how the operation takes place: 
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Summary 


This disclosure introduces a novel lighting control scheme with hand gesture operation and provides the 
next level intelligence of light control. In addition, this disclosure facilitates a flexible and friendly 
interface between a device and an end user. Not only can a light be switched ON and OFF based on 
occupancy, but also the lighting performance, such as dimming up/down or color tuning, can be 
adjusted with user's hands or the body movement without physically touching a control device. This 
would be even more beneficial due to hygiene concerns of the current COVID-19 pandemic 
environment. 
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